THE RELATIONSHIP BETWEEN COMMON FEATURE SELECTION METRICS AND ACCURACY
- Project Research
- 1-5 Chapters
- Abstract : Available
- Table of Content: Available
- Reference Style: APA
- Recommended for : Student Researchers
- NGN 5000
Abstract
Filter-based feature selection methods such as information gain, Gini index, and gain ratio are commonly used in machine learning. It is often assumed that these methods select the most accurate features, but we show this is not true. In this thesis, we study cases when these feature selection metrics and accuracy show “misorderings”: given a pair of features F1 and F2, where F1 has a higher accuracy than F2, the feature selection value is higher for F2 than F1. We first study the frequency of misorderings in randomly-produced synthetic data. Secondly, we study the potential for misordering as two key parameters of the features in a dataset are varied. Finally, we study misorderings in real data and show that misorderings are also prevalent there. Based on our results, we observe that different metrics exhibit different misordering rates, and imposing redundancy-elimination criteria may have the side effect of reducing misordering.
